Investigate wrong predictions

This Notebook is used to analyse wrong predictions. A model is loaded then the confusion matrix is calculated with the models predictions. The confusion matrix is used to determine which classes and the wrong predictions for that class should will be checked.

Then wrong predictions are plotted and saved as images.

Load and Predict

Load

Model and Data

Predict with the model

Prediction of the model over the whole test_set.

Create cunfusion matrix

Analyse wrong predictions

In order to understand why the model predicts wrong, the wronly predicted data is looked at.
First for each class the two most commonly wrong predictions are determined and an accuarcy per class is calculated. This is done by looking at the confusion matrix and each entry for each row (class).

Plot accuracy per class

Get biggest error relative to class size

In order to narrow the examination down to the most important issues, the ratio between each class and its most common false prediction is calculated. If the ratio is equal or greater than 10% then, for this specific combination of class and wrong prediction (this entry in the confusion matrix), the images are plotted to be looked at.

Save Images of biggest Issues

For each entry in the confusion matrix (pair of class and wrongly predicted class) that should be investigated:

The images are then saved to be looked at later.
For the images that just compares to car types, the first two columns are type 1 and the last two columns are type 2.

Class Accuracy (highlightes issues)

Plot accuracy per class again but with highlighted classes of the worst performance. Red are the classes in which a single entry in the confusion matrix make up 10% or more of the errors.

The yellow classes are also underperforming (below 80% accuracy) but instead of a single dominant entry it is multiple entries in the confusion matrix.

Get wrong predicted images for specific classes

Same as above but now the classes are choosen manually, to investigate the yellow highlighted classes from the plot above.

show the images

Get some examples for given class to compare cars